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Abstract 

Recently, Cygan, Kowalik, and Wykurz [IPL 2009] gave sub-exponential-time approximation 
algorithms for the Set Cover problem with approximation ratios better than Inn. In light 
of this result, it is natural to ask whether such improvements can be achieved using lift-and- 
project methods. We present a simpler combinatorial algorithm which has nearly the same 
time-approximation tradeoff as the algorithm of Cygan et at, and which lends itself naturally 
to a lift-and-project based approach. 

At a high level, our approach is similar to the recent work of Karlin, Mathieu, and Nguyen 
[IPCO 2011], who examined a known PTAS for Knapsack (similar to our combinatorial Set 
Cover algorithm) and its connection to hierarchies of LP and SDP relaxations for Knapsack. 
For Set Cover, we show that, indeed, using the trick of "lifting the objective function", we 
can match the performance of our combinatorial algorithm using the LP hierarchy of Lovasz and 
Schrijver. We also show that this trick is essential: even in the stronger LP hierarchy of Sherali 
and Adams, the integrality gap remains at least (1 — e)lnn at level Q(n) (when the objective 
function is not lifted). 

As shown by Aleknovich, Arora, and Tourlakis [STOC 2005], Set Cover relaxations stem- 
ming from SDP hierarchies (specifically, LS+) have similarly large integrality gaps. This stands 
in contrast to Knapsack, where Karlin et al. showed that the (much stronger) Lasserre SDP 
hierarchy reduces the integrality gap to (1 + e) at level 0(1). For completeness, we show that 
LS+ also reduces the integrality gap for Knapsack to (1 + e). This result may be of independent 
interest, as our LS+-based rounding and analysis are rather different from those of Karlin et al., 
and to the best of our knowledge this is the first explicit demonstration of such a reduction in 
the integrality gap of LS + relaxations after few rounds. 
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1 Introduction 



The Set Cover problem is one of the most fundamental and well-studied problems in approxi- 
mation algorithms, and was one of Karp's original 21 NP-complete problems |20j. It can be stated 
quite plainly: given a finite set X of n items and a collection S C 2 X of m subsets of X called 
"cover-sets", the Set Cover problem on instance (X,S) is the problem of finding the smallest 
collection C of cover-sets in S such that X = Usee That is, every item in X must appear in 
at least one cover-set in C. Here, we will consider the minimum cost (or weighted) version of the 
problem, where each cover-set S has a nonnegative cost c(S), and the goal is to find a collection of 
cover-sets with minimum total cost, subject to the above constraint. 

As is well known, the problem can be approximated within a logarithmic factor. For instance, 
Johnson [18] showed that for uniform costs, the simple greedy algorithm that iteratively chooses 
the cover-set containing the maximum number of uncovered elements gives an i? n -approximation 
(where H n = Inn + O(l) is the n'th harmonic number X)i=i V^)- Later, Lovasz [22] showed that 
the cost of the solution found by this greedy algorithm is at most an i/ n -factor larger than the 
optimum value of the natural linear programming (LP) relaxation. 

Chvatal |10j extended these results to a greedy algorithm for the weighted case. He also proved 
that the approximation guarantee of this algorithm is actually only Hf, where b is the size of the 
largest cover-set in S, and moreover that this algorithm gives an f/b-factor approximation relative to 
the optimum of the natural LP relaxation (thus extending the integrality gap bound of Lovasz |22j). 
Slavfk [30] refined the lower-order terms in the analysis of the greedy algorithm's approximation 
guarantee, giving a tight bound of Inn — lnlnn + 0(l). Srinivasan |31] also improved the lower-order 
terms in the upper bound on the integrality gap. On the other hand, the integrality gap of the 
standard LP relaxation for Set Cover is at least (1 — o(l))lnn. Recently, Cygan, Kowalik, and 
Wykurz [11] demonstrated that Set Cover can be approximated within (1 — e) • Inn + 0(1) in time 
2n s +0(iogm)_ j£ - g interesting to note that this time-approximation tradeoff is essentially optimal 
assuming Moshkovitz's Projection Games Conjecture [24] and the Exponential Time Hypothesis 
(ETH) 02]. 

From the hardness perspective, Feige [12] showed that for every constant e > 0, there is no 
(1 — e) In n-approximation algorithm for Set Cover unless all problems in NP can be solved 
deterministically in time n°( loglogn ). To date, the strongest hardness of approximation for Set 
Cover assuming only P ^ NP gives a c In n-hardness for c ~ 0.2267 (Alon et al. [2]). 

1.1 Hierarchies of convex relaxations and connection to Knapsack 

One of the most powerful and ubiquitous tools in approximation algorithms has been the use of 
mathematical programming relaxations, such as linear programming (LP) and semidefinite pro- 
gramming (SDP). The common approach is as follows: solve a convex (LP or SDP) relaxation for 
the 0-1 program, and "round" the relaxed solution to give a (possibly suboptimal) feasible 0-1 
solution. Since the approximation ratio is usually analyzed by comparing the value of the relaxed 
solution to the value of the output (note that the 0-1 optimum is always sandwiched between these 
two), a natural obstacle is the worst case ratio between the relaxed optimum and the 0-1 optimum, 
known as the integrality gap. 

While for many problems, this approach gives optimal approximations (e.g., Raghavendra |26] 
shows this for all CSPs, assuming the Unique Games Conjecture), there are still many cases where 
natural LP and SDP relaxations have large integrality gaps. This limitation can be circumvented 
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by considering more powerful relaxations. In particular, Sherali and Adams |28j Lovasz and Schri- 
jver [23], and Lasserre [21] each have devised different systems, collectively known as hierarchies or 
lift-and-project techniques, by which a simple relaxation can be strengthened until the polytope (or 
the convex body) it defines converges to the convex hull of the feasible 0-1 solutions. It is known 
that, for each of these hierarchies, if the original relaxation has n variables, then the relaxation 
at level t of the hierarchy can be solved optimally in time n°^. Thus, to achieve improved ap- 
proximations for a problem in polynomial (resp. sub-exponential time), we would like to know if 
we can beat the integrality gap of the natural relaxation by using a relaxation at level O(l) (resp. 
o{nj log n)) of some hierarchy. 

Initially, this approach was refuted (for specific problems) by a long series of results showing 
that integrality gaps do not decrease at sufficiently low levels (see, e.g. (HHCEIIE]). Positive results 
have also recently emerged (e.g. [HJ, |29j [5], [15] ) , where improved approximations were given using 
constant-level relaxations from various hierarchies. For a survey on both positive and negative 
results, see [9]. 

Recently, Karlin et al. |19j considered how the use of LP and SDP hierarchies affects the inte- 
grality gap of relaxations for Knapsack. This is of particular relevance to us, since their approach 
relies on a well-known PTAS for Knapsack which is similar in structure to our own combinatorial 
algorithm for Set Cover. They showed that, while the Sherali-Adams LP hierarchy requires Q(n) 
levels to bring the integrality gap below 2 — o(l), level k of the Lasserre SDP hierarchy brings the 
integrality gap down to 1 + 0(1/ A;). While we would like to emulate the success of their SDP- 
hierarchy-based approach (and give an alternative sub-exponential algorithm for Set Cover), we 
note that for Set Cover Alekhnovich et al. [1] have shown that the SDP hierarchy LS+, due to 
Lovasz and Schrijver, requires f2(n) levels to bring the integrality gap below (1 — o(l)) Inn. Never- 
theless, the comparison is not perfect, since the Lasserre hierarchy is much stronger than LS+. In 
particular, previously it was not known whether LS+ also reduces the integrality gap for Knapsack 
(and indeed, the algorithm of Karlin et al. |19| relied a powerful decomposition theorem for the 
Lasserre hierarchy which does not seem to be applicable to LS+). 

1.2 Our Results 

To facilitate our lift-and-project based approach, we start by giving in Section 13.11 a simple new 
sub-exponential time combinatorial algorithm for Set Cover which nearly matches the time- 
approximation tradeoff guarantee in |llj . 

Theorem 1.1. For any (not necessarily constant) 1 < d < n, there is an H n / d - approximation 
algorithm for Set Cover running in time poly(n,m) • m 0<yd \ 

While this theorem is slightly weaker than the previous best known guarantee, our algorithm 
is remarkably simple, and will be instrumental in designing a similar lift-and-project based Set 
Cover approximation. The algorithm is combinatorial and does not rely on linear programming 
techniques. By choosing d = n £ , we get a sub-exponential time algorithm whose approximation 
guarantee is better than Inn by a constant factor. 

Next in Section 13. 2\ we show that using level d of the linear programming hierarchy of Lovasz- 
Schrijver [23j, we can match the performance of the algorithm we use to prove Theorem ll.il though 
only if the "lifting" is done after guessing the value of the objective function (using a binary search), 
and adding this as a constraint a priori. In this case, the rounding algorithm is quite fast, and avoids 
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the extensive combinatorial guessing of our first algorithm, while the running time is dominated by 
the time it takes to solve the LP relaxation. 

On the other hand, without the trick of "lifting the objective function", we show in Section [J] 
that even the stronger LP hierarchy of Sherali-Adams [28] has an integrality gap of at least (1— e) In n 
at level O(n). Specifically, we show the following 

Theorem 1.2. For every < e, 7 < \, and for sufficiently large values of n, there are instances 
of Set Cover on n cover-sets (over a universe of n items) for which the integrality gap of the 

level- \ 7 ^f 1 6 - n I Sherali-Adams LP relaxation is at least TT^lnn. 

As we have mentioned, the prospect of showing a positive result using SDP hierarchies is unlikely 
due to the work of Alekhnovich et al. [1] which gives a similar integrality gap for LS + . 

For completeness, we also show in Section [5] that the Lasserre-hierarchy-based Knapsack al- 
gorithm of Karlin et al. |19] can be matched using the weaker LS+ hierarchy and a more complex 
rounding algorithm. Specifically, we show that the integrality gap of the natural relaxation for 
Knapsack can be reduced to 1 + e using 0(e~ 3 ) rounds of LS+. This highlights a fundamental 
difference between Knapsack and Set Cover, despite the similarity between our combinato- 
rial algorithm and the PTAS for Knapsack on which Karlin et al. rely. Formally, we prove the 
following: 

Theorem 1.3. The integrality gap of level k of the LS+ relaxation for Knapsack is at most 

1 + 0(fc-V3). 

In what follows, and before we start the exposition of our results, we present in Section [2] the 
Lovasz-Schrijver system along with some well-known facts that we will need later on. We end with 
a discussion of future directions in Section [6l 

2 Preliminaries on the Lovasz-Schrijver System 

For any polytope P, this system begins by introducing a nonnegative auxiliary variable xq, so that 
in every constraint of P, constants are multiplied by xq. This yields the cone Kq(P) := {(xq, xqx.) \ 
xo > & x € P}. For an n-dimensional polytope P, the Lovasz-Schrijver system finds a hierarchy 
of nested cones K Q {P) D K-i(P) 5 • • • 5 K n (P) (in the SDP variant, we will write Kf(P)), defined 
recursively, and which enjoy remarkable algorithmic properties. In what follows, let Vk denote the 
space of vectors indexed by subsets of [n] of size at most k, and for any y G Vk, define the moment 
matrix to be the square matrix with rows and columns indexed by sets of size at most \_k/2\, 
where the entry at row A and column B is yAuB- Also we denote by eo,ei, . . . ,e n the standard 
orthonormal basis of dimension n + 1, such that Y^ej is the i-th. column of the moment matrix. 

Definition 2.1 (The Lovasz-Schrijver (LS) and Lovasz-Schrijver SDP (LS+) systems). Consider 
the conified polytope Kq(P) defined earlier (let us also write Kq(P) = Kq(P)). The level-t 
Lovasz-Schrijver cone (relaxation or tightening) Kt(P) (resp. K^(P)) of LS (resp. LS+J is re- 
cursively defined as all n + 1 dimensional vectors (xrj,xo x ) f or which there exist y 6 V2 such 
that F^e^yM (e - e») € #t-i(P) (resp. K+^P)) and (x ,x x) = Y^e . The level-t Lovasz- 
Schrijver SDP tightening of LS+ asks in addition that is a positive-semidefinite matrix. 

In the original work of Lovasz and Schrijver |23| it is shown that the cone K n {P) (even in the 
LS system) projected on xq = 1 is exactly the integral hull of the original LP relaxation, while one 
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can optimize over Kt(P) in time n°^\ given that the original relaxation admits a (weak) polytime 
separation oracle. The algorithm in this section, as well as the one in Section [5j both rely heavily 
on the following facts, which follow easily from the above definition: 

Fact 2.2. For any vector x G [0, l] n such that (l,x) G Kt(P), and corresponding moment vector 
y G V2, and for any i G [n] such that Xi > 0, the rescaled column vector ^Y^ej is in Kt-i(P) PI 
{(l,x') |x'G [0,1]"}. 

Fact 2.3. For any vector x G [0, l] n suc/i £/iai (l,x) G Kt(P), and any coordinate j such that Xj 
in integral, for all t' < t, any vector x' such that (l,x') G K t '(P) derived from x by one or more 
steps as in Fact \2.2\ we have x'j = Xj. 

3 Two Approaches for Proving Theorem 11.11 

In the following sections, we let (X, S) denote a Set Cover instance with items X and cover-sets 
S where each S G S has cost c{S). We use n to denote the number of items in X and m to denote 
the number of cover-sets in S. 

3.1 Sketch of a Combinatorial Proof 

Recall that the standard greedy algorithm for approximating Set Cover iteratively selects the 
cover-set S of minimum density c(S)/\S \ \J TeC T\ where C is the collection of cover-sets already 
chosen. The approximation guarantee of this algorithm is H^, where b is the size of the largest 
cover-set. Our algorithm builds on this result simply by guessing up to d cover-sets in the optimal 
solution before running the greedy algorithm. However, some of the cover-sets in S that were 
not guessed (and might still contain uncovered items) are discarded before running the greedy 
algorithm. Specifically, we discard the cover-sets that contain more than Q uncovered items after 
initially guessing the d sets. We show that for some choice of d sets, no remaining set in the 
optimum solution covers more than 5 uncovered items. Thus, running the greedy algorithm on the 
remaining sets is actually an i/ n / rf -approximation. The full description of the algorithm along with 
all details of the proof are in Appendix lAl 

3.2 Proof Based on the Lovasz-Schrijver System 

In this section we provide an alternative LP-based approximation algorithm for Set Cover with 
the same performance as in Section 13.11 illustrating the power of the so-called lift-and-project 
systems. Consider the standard LP relaxation 

minimize c(s)xs 
SeS 

subject to ^2 x s > 1 ViGl (1) 

SBi 

< x s < 1 V S G S (2) 

for Set Cover. Now, consider the corresponding feasibility LP where instead of explicitly mini- 
mizing the objective function, we add the following bound on the objective function as a constraint 
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(we will later guess the optimal value q by binary search): 

c(s)x s < q. (3) 

SeS 

We will work with the feasibility LP consisting only of constraints ([1]), ([2]) and ([3|) (and no objective 
function). Denote the corresponding polytope of feasible solutions by P q . 

In what follows we strengthen polytope P q using the Lovasz-Schrijver lift-and-project system. 
Next we show that the level-d Lovasz-Schrijver relaxation Kd(P q ) can give a //"-factor approxi- 
mation algorithm. We note here that applying the Lovasz-Schrijver system to the feasibility P q 
(which includes the objective function as a constraint) and not on the standard LP relaxation of 
Set Cover is crucial, since by Alekhnovich et al. [T] the latter LP has a very bad integrality gap 
even when strengthened by O(n) rounds of LS+ (which is even stronger than LS). 

To that end, let q be the smallest value such that the level-ci LS tightening of P q is not empty 
(note that q < OPT). The value q can be found through binary search (note that in each stage of 
the binary search we attempt to check Kd(P q ') for emptyness for some q', which takes time m°^). 
Our goal is to show that for this q we can find a Set Cover of cost at most q ■ Ha . 

d 

Let x( rf ) be such that (l,x^) € Kd(P q ). For any coordinate i in the support of we can 

invoke Fact EJ and get a vector x^" 1 ) such that (l,x( d_1 )) G K d ^i(P q ) and xf _1) = 1. By 
Fact 12.31 by iterating this step, we eventually obtain a vector x(°) £ P q which is integral in at 
least d coordinates. Note that by constraint ([3|), this solution has cost at most q. We refer to this 
subroutine as the Conditioning Phase, which is realized in d many inductive steps. 

If at some level < i < d, the sets whose coordinates in are set to 1 cover all universe 
elements X, we have solved the Set Cover instance with cost 

C ( S ) < <1 < 0PT - 

S:x« = l 

Otherwise, we need to solve a smaller instance of Set Cover defined by all elements Y C X not 
already covered, using cover- sets T ={Sny[x|? ) > 0}. We introduce some structure in the 
resulting instance (Y, T) of Set Cover by choosing the indices we condition on as in the proof of 
Lemma IA.2I This gives the following Lemma, whose proof is similar to that of Lemma IA.2I 

Lemma 3.1. // at each step of the Conditioning Phase we choose the set S in the support of the 
current solution x.( d > containing the most uncovered elements in X, then for all T € T we have 

Proof. For 1 < i < d let Si denote the cover-set chosen in the Conditioning Phase for level i and 
for < i < d let Ci = {Sd, Sd-i, ■ ■ ■ , Si+i} (with Q = 0). We show that at every iteration 1 < i < d 

we have |5 \ UreCi ^1 — d-l+i ^ or evei T & £ S \C-i with x^ > 0. 

For 1 < i < d, let a» := |<Si\(J TgC . T\. Since we chose the largest (with respect to the uncovered 
items) cover-set Si in the support of xM we have \S' \ IJtgC — ai ^ or evei T cover-set S' G S \ Ci 

(i) 

with x^; > 0. For 2 < i < d we also have aj_i < ai because Ci C d-\ and because we chose Si 
instead of Si-\ in the Conditioning Phase for level i. 

So, Qrf > a,i~\ > ■ ■ ■ > ol\. Now, each item j covered by Cq contributes 1 to Qj for the earliest 
index i for which j £ Si, so Yli=i a i — n - This implies that Qj < for each 1 < i < d. 

Therefore, every set in the support of x(°) has at most ^ elements that are not already covered by 
Cq. So, the instance (Y,T) has |T| < ^ for any T € T. □ 
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Let T> be the collection of cover-sets chosen as in Lemma 13.11 Observe that the vector x(°) 
projected on the cover-sets S\T> that were not chosen in the Conditioning Phase is feasible for the 
LP relaxation of the instance (Y,T). In particular, the cost of the LP is at most q — J2seT> C (S), 
and by Lemma 13. II all cover-sets have size at most ^. By Theorem IA.1( the greedy algorithm will 
find a solution for (Y,T) of cost at most Hii ■ — ^sex> c ('~')) • Altogether, this gives a feasible 
solution for (X, S) of cost 

H i ' ( q ~ C ( S H + 2 c ^ < H %-q< H %- OPT - 
4 Linear Sherali-Adams Integrality Gap for Set Cover 

The level-^ Sherali-Adams relaxation is a tightened LP that can be derived systematically starting 
with any 0-1 LP relaxation. While in this work we are interested in tightening the Set Cover 
polytope, the process we describe below is applicable to any other relaxation. 

Definition 4.1 (The Sherali-Adams system). Consider a polytope over the variables yi,...,y n 
defined by finitely many constraints (including the box- constraints < ?/, < 1). The level-l Sherali- 
Adams relaxation is an LP over the variables {yA} where A is any subset of {1,2, ... ,n} of size 
at most £ + 1, and where y$ = 1. For every constraint Y17=i a *2/« — ^ °f o^gmaZ polytope and 
for every disjoint P,E C {1, . . . , n} with \P\ + \E\ < £, the following is a constraint of the level-l 
Sherali-Adams relaxation 

n 

Y, a i (- l ) m ypuTu{i}>b Yl (-i) |T| ypuT- 

i=l 0CTCB 0CTCE 

We will prove Theorem 1 1.2 1 in this section. For this we will need two ingredients: (a) appropriate 
instances, and (b) a solution of the Sherali-Adams LP as described in Definition 14.11 Our hard 
instances are described in the following lemma, which is due to Alekhnovich et al. [J. 

Lemma 4.2 (Set Cover instances with no small feasible solutions). 

For every e > ij > 0, and for all sufficiently large n, there exist Set Cover instances over a 
universe of n elements and n cover-sets, such that: 

(i) Every element of the universe appears in exactly (e — rf)n cover-sets, and 

(ii) There is no feasible solution that uses less than log 1+£ n cover-sets. 

In order to prove Theorem 11.21 we will invoke Lemma 14.21 with appropriate parameters. Then 
we will define a vector solution for the level-^ Sherali-Adams relaxation as described. 

Lemma 4.3. Consider a Set Cover instance on n cover-sets as described in Lemma \4-%\ Let 
f denote the number of cover-sets covering every element of the universe. For f > 3£, the vector 
y indexed by subsets of {1, . . . , n} of size at most £ + 1 defined as yA '■= rj~g~j^pTji ; C 
{1, . . . ,n}, \A\ < £+1, satisfies the level-£ Sherali-Adams LP relaxation of the Set Cover polytope. 

The proof of Lemma [4.3l involves a number of extensive calculations which we give in Section P4. 1 1 
Assuming the lemma, we are ready to prove Theorem 11.21 
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Proof of Theorem \1.2l Fix e > and invoke Lemma 14.21 with r] = e 2 to obtain a Set Cover 
instance on n universe elements and n cover-sets for which (i) every universe element is covered 
by exactly {e — e 2 )n cover-sets, and (ii) no feasible solution exists of cost less than log 1+e n. Note 
that in particular (i) implies that in the Set Cover LP relaxation, every constraint has support 
exactly / = (e — e 2 )n. 

Set £ = 1+1 n and note that f /£ > 3, since 7 < \. This means we can define a feasible level- £ 
Sherali-Adams solution as described in Lemma 14.31 The values of the singleton variables are set to 

- 1 _ 1+7 

V{i} ~ {e-e 2 )n-£~ (e-e 2 )n 

But then, the integrality gap is at least 

OPT e-e 2 , e-e 2 
z2i=iV{i} 1 + 7 (l + 7)ln(l + e) 

The lemma follows once we observe that ln(l + e) = e — ^e 2 + 0(e 3 ). □ 
4.1 Proof of Lemma 14.31 

Recall that / denotes the support of every cover constraint in the Set Cover relaxation (i.e. the 
number of cover-sets every element belongs to in the instance described in Lemma l4.2p . Also recall 
that 

To show feasibility of the above vectors, we need to study two types of constraints, i.e. the so called 
box-constraints 

0< Yl (- l ) m VPuT<l, VP,£C{l,..,n}, \P\ + \E\<£+\, (4) 

0CTCE 

as well as the covering constraints 

Y, E i-V) m VPimm > E (-l) lT{ yPuT,VP,EC{l,...,n}, \P\ + \E\<£, (5) 

ieDQCTCE 0CTC£ 

where D C {1, . . . ,n} is a set of / many cover-sets covering some element of the universe. The 
symmetry of the proposed solutions allows us to significantly simplify the above expressions, by 
noting that for \P\ = p and \E\ = e we have 



E (-i) |t| ^ut = E(- 1 ) < C) 

-w(- ep +— n V / 



0CTC£ t=0 



(f-£-l+p + t)\ 



For the sake of exposition, we show that our LP solution satisfies the two different kinds of 
constraints in two different lemmata. Set x = f — I — 1 + p, and note that if / > 3£, then since 
e < £ we have x > |e. Thus, the feasibility of the box constraints dH) (for our solution) is implied 
by the following combinatorial lemma. 
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Lemma 4.4. For x > |e we have 



1 

< 



+ t)\ (x-p) 



Proof. First we show the lower bound, for which we study two consecutive summands. We note 
that 

1 ( e \ 1 fe\ 1 / ( 2 ^) 1 \ 



2i/(x + 2f)! V 2 * + V (a; + 2i + l)! V 2 V + 2 *) ! V (It) x + 2t + 1 

e\ 1 / e-2t 1 



> 



It) (x + 2t)\ V 2i + 1 x + 2t + 1 
e\ 1 / e 1 



2ty (x + 2t)! V 2t + lx + 2t + l 



~ \2t) (x + 2t)\ i 1 x 

Since x > e, every two consecutive summands add up to a non-negative value. Thus, if the number 
of summands is even (that is, if e is odd), the lower bound follows, and if the number of summands 
is odd, the bound also follows, since for even e, the last summand is positive. 

Now we show the upper bound. Note that, since p does not appear in the sum, it suffices to 
bound the sum by the smaller value -=j. This is facilitated by noting that 



^ { ' \t {x + t)\ x\ ^ { ' t\ ( x+e Y 



Hence, it suffices to show that 



r-v 1 1 fx + e\ / x + e 



As before, we analyze the sum of two consecutive terms (this time in the above sum). This is done 
in the next claim. 

Claim 4.5. For x > |e we have 

1 / x + e\ ( x + e \ ( x + e\ ( x + e 



(2i)! \x + 2tj (2t + 1)! \x + It + I J ~ \x + 2t J \x + 2t + l y 
Proof. We divide both sides of the desired inequality by (^T^t) *° obtain the equivalent statement 



1 1 e-2t e-2t 

< 1 



(2t)\ (2t + 1)! x + 2t + 1 ~ x + 2t + 1 
e-2t ( 1 \ 1 



x + 2t + 1 V (2t + l)!/ - (2t)! 

Note that the above is tight for i = 0. For t > 0, and since x > |e, we have x ^.2^+i ^ f wm ch is 
small enough to compensate for the worst-case ratio of the expressions involving factorials, which 
occurs for t = 1. □ 
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Continuing our proof of Lemma 14.41 first suppose that e is odd. Then Claim |4"31 implies that 



t=0 \ 1 / t=0 



which gives the required condition © for odd e. If e is a positive even integer, then again Claim |4"3 
implies that 

t=0 v 7 t=0 \ 1 / \ / V 

Finally, if e = then © holds with equality. This concludes the proof. □ 

Now we turn our attention to cover constraints ([5]). Recall that the values i/a only depend on 
the size of A. Since / and I are fixed in the context of Lemma 14.31 for |P| = p and |P| = e we can 
define 

(/-*-!)! 



0CTCE t=0 ^ 7 



(f-e-i+p + ty: 



Thus the left-hand-side of ([5]) (for fixed P and P) involves only expressions of the form P e ,p, P ejP +i, 
or 0, depending on the relationship between the sets P, E and {i}, while the right-hand-side equals 
H e , P - 

More concretely, let \D fl P| = pi, \D fl P| = e±, \P \ D\ = po and \E \ D\ = eo, where 
Po + Pi — \P\ = p and eo + ei = \E\ = e, and recall that E and P are disjoint. Then observe 
that the p\ indices in D n P each contribute -f^ e ,p to the left-hand-side of ©. In addition, the e\ 
indices i G D fl P each contribute to the left-hand-side. This follows because each T C P \ {i} 
can be paired with T" := T U {i} C P and the terms 2/putu{«} an d 2/puT'u{j} are identical, while 
they appear in the sum with opposite signs. Finally, the remaining j — p\ — e\ indices contribute 
each P ejP+ i. Overall, for our proposed solution, Constraint ([5]) can be rewritten as 

(/ ~Pi ~ ei)P e , p+ i +piH e:P > H C)P 

Clearly, for p\ > 0, the above constraint is satisfied. Hence, we may assume that |P D D\ = 0, and 
so |P| = p\ = p. Note also that the value e\ does not affect P ejP , thus the above inequality holds 
for all ei < e iff it holds for e± = e and eo = 0. To summarize, to show that Constraint ([5]) is 
satisfied, we need only show that 

(/ - e)P e , p+1 > P e , p (7) 

for H e , p = Et=o(- 1 ) t (t) (/Ifci+p-H)! ■ and e+p < f. Note that the value (/ - i - 1)! in the 
numerator appears in both sides. Also / > 31 and e < t implies that e < 21 < f — £. Finally recall 
that in © we have p + e = |P| + \L\ < £. Hence, to show that ((7|) (and thus Constraint ([5])) is 
satisfied, it remains only to show the following lemma. 

Lemma 4.6. For e < / — I and p + e < I we have 

</ - •) d-d* (:) w^Tw. - £<->' (:) {f . e .i +P+t , >- o 
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Proof. We rewrite the left-hand-side as 

1 / f-e 



t=o v 7 



1 



(f -e-i+ P + ty. \f-£+ P + t 

1 £-(p + e)-t 

(f-£-i + p + ty. f-e+p + t 



£ ( - 1)l C) 

(^-^e, ) |:,- 1 ,'(;) I7 3 7 ^-|:(- 1 )«(;) 



(/-* + p + t)! 



The first sum of the last expression is non-negative, as it is a multiple of the box-constraints we 
have already proven. So is its coefficient, since p + e < I. Note that for e = and p = £ the above 
expression equals (and in particular Constraint ([5]) is tight). Next we show that for other values 
of p, e (either p < £ or e > 0) the above expression is strictly positive. For this it suffices to show 

thatELo(-l) t (t) I 7=lfeH]T<0. 

We proceed again by analyzing every two consecutive terms. We observe that for t > 1 we have 

e \ It - 1 ( e \ 2i 

+ 



2t - 1/ (/ - £ + f> + 2i - 1)! \2t) {f - £ + p + 2t) 



e 



(/- 


+ p + 2i - 


1)! 




(24-1) 




(/-■ 


+ p + 2t - 


1)! 




\2t-1) 




(/- 


t + p + 2t- 


1)! 



1 of 1 ) , \ e - 2t + 1 

-(2t - 1) + 



< 77 ; o, ,m -( 2i " 1) + 



/ - ^ + p + 2t 

e 



f 



f- 

which is non-positive, since / — £ > e. This argument shows that every two consecutive summands 
(f-il p+ t)\ ad( i up to a non-positive value. If e is even, then we are done, while for 
odd e the unmatched summand (for t = e) is negative, and so the lemma follows. □ 



5 An LS + -based PTAS for Knapsack 

We consider the Knapsack problem: We are given n items which we identify with the integers [n] , 
and each item i E [n] has some associated (nonnegative) reward and cost (or size) c%. The goal is to 
choose a set of items which fit in the knapsack, i.e. whose total cost does not exceed some bound C, 
so as to maximize the total reward. In what follows we will use the LP {max X^ILi r i x i '■ S?=i c *- r « — 
C&0<Xi<lVi£ [n]}, which is the natural relaxation for Knapsack. 

Denote the polytope associated with the Knapsack LP relaxation by P. We will consider the 
SDP derived by applying sufficiently many levels of LS+ (as defined in Section |3.2|) to the above 
LP. That is, for some £ > 0, we consider the SDP 



maximize 




i=l 

subject to (l,x) G Kj(P). 
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There is a well-known simple greedy algorithm for Knapsack: Sort the items by decreasing 
order of rj/cj, and add them to the knapsack one at a time until the current item does not fit. The 
following lemma (which is folklore) relates the performance of the greedy algorithm to the value of 
the LP relaxation P: 

Lemma 5.1. Let x be a solution to P, and Rq be the reward given by the greedy algorithm. Then 
Yji r i x i < R G + maxj n. 

This gives a trivial bound of 2 on the integrality gap, assuming that Cj < C for all i (that is, 
that each item can be a solution on its own), since we then have i?G + maxj rj < 2OPT0 We will see 
later that the above assumption can essentially be enforced, that is, that we can ignore items with 
reward greater than C (see Lemma l5.6p . Lemma 15, II has the following easy corollary: Consider the 
above greedy algorithm, with the modification that we first add all items which have Xi = 1 and 
discard all items which have Xi = 0. Then the following holds: 

Corollary 5.2. Let x be a solution to P, and let R' G be the total reward given by the above modified 
greedy algorithm. Then rjXj < R' G + maxj : o< x -i<i I- 

We will show that, for any constant e > 0, there is a constant L £ such that the SDP relaxation 
for Knapsack arising from level L £ of LS_|_ has integrality gap at most 1 + 0(e). For the Lasserre 
hierarchy, this has been shown for level 1/e [19]. We will show this for L E = 1/e 3 in the case of 
LS+. 

Our rounding algorithm will take as input the values of the Knapsack instance (rj)j, (cj)j, and 
C, an optimal solution x s.t. (1, x) £ K^~(P) (for some level i > 0, initially I = L £ ), and parameters 
e and p. The parameter p is intended to be the threshold e-OPT in the set SeOPT = {i \ t% > e-OPT}. 
Rather than guessing a value for OPT, though, we will simply try all values of p 6 {ri \ i € [n]}U{0} 
and note that for exactly one of those values, the set 5" e oPT coincides with the set {i | r,, > p} (also 
note that p is a parameter of the rounding, and not involved at all in the SDP relaxation). 

The intuition behind our rounding algorithm is as follows: As we did for Set Cover, we would 
like to repeatedly "condition" on setting some variable to 1, by using Fact 12.21 If we condition 
only on (variables corresponding to) items in S^opt, then after at most 1/e iterations, the SDP 
solution will be integral on that set, and then by Corollary 15.21 the modified greedy algorithm 
will give a 1 + O(eOPT) approximation relative to the value of the objective function (since items 
outside <S £ opt have reward at most eOPT). The problem with this approach (and the reason why 
LP hierarchies do not work), is the same problem as for Set Cover: the conditioning step does 
not preserve the value of the objective function. While the optimum value of the SDP is at least 
OPT by definition, after conditioning, the value of the new solution may be much smaller than 
OPT, which then makes the use of Corollary 15.21 meaningless. The key observation is that the 
use of SDPs ensures that we can choose some item to condition on without any decrease in the 
objective function (see Lemma l5.4p FI A more refined analysis shows that we will be able condition 
specifically on items in S^opt without losing too much. Counter-intuitively, we then need to show 
that the algorithm does not perform an unbounded number of conditioning steps which increase the 
objective value (see Lemma [5.8p . Our rounding algorithm KS-Round is described in Algorithm[TJ 
while the performance guarantee is described in Section [5.11 

1 Here, as before, OPT denotes the optimal 0-1 solution. 

2 Note that this is crucial for a maximization problem like Knapsack, while for a minimization problem like Set 
Cover it does not seem helpful (and indeed, by the integrality gap of Alekhnovich et al. [T], we know it does not 
help). 
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Algorithm 1 KS-Round((rj)j, (cj)j, C, x, e, p) 



Let y G V2 be the moment vector associated with (l,x). 
Let S p ^— I > p}, and let 5 6 <— {z | Xj = 6} for 6 = 0, 1. 
if S P CS°U S 1 then 



Run the modified greedy algorithm. 

n 

5: else if r^Xj < e • r^j then 

ies p \s 1 i=i 
6: Run the modified greedy algorithm on items in ([n] \ S^) U 5 . 

n n 

7: else if there is some i G S p \ (5° U S 1 ) s.t. ^ r jV{i,j} ^ (1 — e 2 )^ ■ ^ then 

j=i 3=1 
8: Run KS-Round((rj)i, (a)i, C, ^Y^e h e, p). > See Fact 

9: else 

n n 

10: Choose i G [n] \ (S p U S ) s.t. r jy{i,j} > (1 + s 3 )^ ' '"i^j > See Lemma 

3=1 3=1 
11: Run KS-Round((rj)i, (cj)*, C, ^Y^e;, e, p). > See Fact 

12: end if 



5.1 Analysis of Algorithm [T] 

Before we analyze the performance guarantee of the algorithm, there is one more simple fact about 
both LS and LS+ which we will use in this section. 

Fact 5.3. Given a solution (l,x) G Ki(P) for some t > 1 and corresponding moment vector 
y G V 2 , ifxi = 1 for some i G [n], then Y^e t = Y^e (= (l,x)). 

This fact follows easily from the fact that (eo — ej) G Kt-i(P), and since if a;, = y^y = 1 
then the first entry in the above vector is 0, which for the conified polytope Kt—\(P) only holds for 
the all-zero vector. 

Now, consider the set S p defined in Step[2j As we have pointed out, for the appropriate choice 
of p, this set coincides with the set S^opt- Note that, by Corollary 15.21 if all the X{ values in S p 
are integral, then Step 0] returns a solution with value R' G satisfying 

OPT > R! g > r % x % ~ max r i > riXi ~ £ ' ®^ '■ 

i i^Sp . 

Now, if x is the original SDP solution given to the rounding algorithm, this gives an upper bound 
of 1 + e on the integrality gap (as well as a (1 + e)-approximation). However, in Steps [HI and II 11 
we recurse with a new SDP solution (to a lower level in the LS+ hierarchy). Thus, our goal will 
be to arrive at an SDP solution which is integral on S p (or assigns so little weight to S p that we 
can ignore it), but we need to show that the objective function does not decrease too much during 
these recursive steps. To show this, we will rely crucially on the following easy lemma, which is 
also the only place where we use positive-semidefiniteness. 

Lemma 5.4. Let (l,x) be a solution to K^~(P) for some I > 1, with the corresponding moment 
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vector y. Then the solution satisfies 

n n / n \ ^ / n n 

Y n Y r iy{hj} ^ Y riXi [ = Y ri Y Xir f 

i=l j=l \i=l ) \ i=l j=l 

Proof. By the positive semidefiniteness of the moment matrix Y^, we have a T yt y la > for any 
vector a € M n+1 . Then the lemma follows immediately from this inequality, by letting a = (cij)j be 
the vector defined by a% = for i S [n] and do = — Y^i=i r i x i- ^ 

Thus, there is some item i £ [n] on which we can condition (by taking the new solution -^-Y^ei, 
without any decrease in the value of the objective function. Moreover, using the above lemma, 
we can now show that the algorithm is well-defined (assuming we have start with an SDP at a 
sufficiently high level of LS+ for all the recursive steps). 

Lemma 5.5. If Step{W[ is reached, then there exists an item £ € [n] \ (S p U S°) satisfying 

n n 

Y r iy{iJ} >( 1 + £3 ) x i ■ Y r i x r ( 8 ) 

3=1 j=l 

Proof. Note that if Step [10] is reached then we must have 

n 

Y r iXi > £ -Y r i X i- ( 9 ) 

iGSpXS 1 i=l 

n n 

Moreover, for all i G S p \ (S° U S 1 ) we have Y2 r iy{ i >j} < (1 — £ 2 )xi ■ Y2 r i x i' ^ n P ar ticular, we 
have 

n n 

Y r *Y r iy{ij} < ( x - e2 ) Y TiXi ■ Y r i x r ( 10 ) 

ieSp^ 1 j=i icSpXS 1 j=i 
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Thus (noting that all i £ S° contribute nothing to the following sums), we have 



n n 



ii 



*6([n]\(S p US°))US' 1 j=l i=l 3=1 ie5 p \5 1 3=1 

Cn \ ^ n 

Y riXi ) ~ Y^ ri Y r 3 y {id} by Lemma E 

i=l J i&SpXS 1 3=1 

> Y nxi - - e2 ) Y nxi ' Y r 3 x i by *M) 

\i=l / ieS p \S 1 3=1 

n n 
— /jXj ^ijXjtt /jXj 3 X 3 

*e(N\5'p)u5 1 j=i iGSpV^ 1 j=i 

n n n 

> Y r i X i ' Y^ T 3 X 3 + ^ X] r * Xi ' Ys T 3 X 3 by @ 
iG([n]\5'p)U5 1 3=1 i=l 3=1 

n 

>(l + e 3 )- Y riXi-Y^fjXj. 

«6(M\5'p)US 1 3 = 1 

Therefore, there is some i € ([n] \ (5 P U 5°)) U S 1 satisfying (jSJ). We only need to show that 

i £" S , that is, that Xi < 1. However, by Fact 15.31 if = 1 then = £j, making inequality (jHJ) 

impossible for such i. □ 

Now that we have shown the algorithm to be well-defined, let us start to bound the depth of 
the recursion. We will use the following lemma to show that after a bounded number of recursive 
calls in StepEl the SDP solution becomes integral on S p : 

Lemma 5.6. Let (l,x) be a solution to Kf for some i > 1. Then for all items i £ [n] \ S l such 
that Ci > C — SjgS 1 c j> we h- ave x i = 0- 

Proof. Suppose, for the sake of contradiction, that there is some item i satisfying this property, 
with < Xi < 1. By Fact 15.31 f° r every j £ S 1 we have yuj\ = Xi- However, by Fact 12.21 the 
vector j-y[ y lej satisfies the constraints of K^_ v and in particular the capacity constraint. But this 
is a contradiction, since we have 

n 

k Y C 3 y {iJ} r ' • Y ''' ' Y C 3 X 3 > C - 

3=1 JGS 1 je[n]\(S!u{i}) 

□ 

Since no 1/e items from 5 e oPT can fit simultaneously in the knapsack, we have the following: 

Corollary 5.7. For p s.t. S p = S £ opj, the recursion in Step cannot be repeated times. 
Moreover, if this step is repeated ([1/e] — 1) times, then after these recursive calls, we have Xi £ 
{0,1} for alii £ S p . 

We can now bound the total depth of the recursion in the algorithm. 
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Lemma 5.8. For all positive e < 0.3, the algorithm performs the recursion in Step [77] less than 
[1/e 3 ] times. 

Proof. This follows by tracking the changes to the value of the objective function. Each time Step [8] 
is performed, the value of the objective function changes by a factor at least (1 — e 2 ), while each 
time Step [TT] is performed, this value changes by a factor greater than (1 + e 3 ). Assuming, for 
the sake of contradiction, that Step [11] is performed [l/e 3 ] times, then by Corollary 15.71 the total 
change is at least a factor (1 — £ 2 )^ 1 / £ T~ 1 (1 + £ 3 )l" 1//£ 1 > 2. However, this is a contradiction, since 
initially the value of the objective function is at least OPT, and the integrality gap is always at 
most 2. □ 

Finally, we can prove that the algorithm gives a (1 + O (^-approximation, thus bounding the 
integrality gap. Theorem 11.31 follows immediately from the following. 

Theorem 5.9. For sufficiently small e > 0, given an optimal (maximum feasible) solution (l,x) 
to Kg~(P) for I = 0(l/e 3 ), there is a value p € {r{ | i G [n]} U {0} such that algorithm KS-Round 
finds a solution to the Knapsack instance with total value (reward) at least (1 — 0(e)) ■ X^iLi r i x i- 

Proof of Theorem \5.9l It is easy to see that for (exactly one) p € {Yj | i € [n]} U {0} we have 
S p = S £ opj. Choose this value of p. 

Note that the only time the value of the objective function can decrease is in Step [8] Let <I> be 
the initial value of the objective function. Then by Corollary 15.71 at the end of the recursion, the 
new value of the objective function will be at least (1 — e 2 ) 1 ^ • $ = (1 — 0(e))$. Thus, it suffices 
to show the theorem relative to the final value of the objective function (as opposed to *g 

Let us consider the algorithm step-by-step. If the algorithm terminates in StepH] then since for 
all i G [n] \ 5 £ opt we have r j < e • OPT, by Corollary 15.21 the rounding loses at most a (1 + 0(e)) 
factor relative to the value of the objective function. 

If the algorithm terminates in Step [6] then similarly, the rounding loses at most a (1 + 0(e))- 

factor relative to J2ie[n]\s p r i x i > i 1 ~ £ ) Ya=i r i x i- 

Finally, note that by Corollary 15.71 and Lemma 15.81 we have sufficiently many levels of the 
hierarchy to justify the recursive Steps l8l and I 111 and note that the choice of item i in Step 1101 is 
well-defined by Lemma 15.41 □ 



6 Conclusion 

The known sub-exponential time approximation algorithms place Set Cover in a distinct category 
from other optimization problems like MAX-3SAT in the following sense. Though one can achieve 
provably hard approximation factors for Set Cover in sub-exponential time, Moshkovitz and 
Raz |25j show that improving over the easy ^-approximation for MAX-3SAT [18] by any constant 

requires time 2 nl ° (1> , assuming the ETH. 

Rather, Set Cover lies in the same category of problems as Chromatic Number and Clique 
both of which admit n 1 " 6 approximations in time 2 0( - n ~^ (by partitioning the graph into sets of 
size n £ and solving the problem optimally on these sets), despite the known n 1 " ^ hardness of 

3 In fact, this will also show that the integrality gap remains at most l + 0(e) throughout the algorithm (as opposed 
to 2, which we used in the proof of Lemma |5.8[) . thus bounding the depth of the recursion by 0(l/e 2 ) rather than 

0(l/£ 3 ). 
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approximation for both problems |16 t 113 1 f32] . This may also be taken as evidence that the recent 
subexponential time algorithm of Arora, Barak, and Steurer [3] for Unique Games does not 
necessarily imply that Unique Games is not hard, or not as hard as other NP-hard optimization 
problems. 

Finally, turning to lift-and-project methods, we note here that our LS+-based algorithm for 
Knapsack shows that in some instances reduced integrality gaps which rely heavily on properties 
of the Lasserre hierarchy can be achieved using the weaker LS+ hierarchy. This raises the question of 
whether the problems discussed in the recent series of Lasserre-based approximation algorithms [5j 
[151 EZ] also admit similar results using LS+. On the flip side, it would also be interesting to see 
whether any such problems have strong integrality gap lower bounds for LS+, which would show a 
separation between the two hierarchies. 
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A A Combinatorial Approximation Algorithm 

The following result is well-known and follows from standard dual-fitting techniques. 

Theorem A.l (Chvatal |10j . paraphrased). The greedy algorithm is a polynomial-time H^- approximation 
where h is the size of the largest cover-set in S. Moreover, it finds a solution whose cost is at most 
times the optimum value of the standard LP relaxation. 

Motivated by the former Theorem, we present next our combinatorial algorithm (sketched in 
Section [3TT]) in full details. 

The outer loop of Algorithm [2] is reminiscent of the steps in many PTASes that "guess" the 
largest items to be used (for example, many Knapsack variants such as in [7j). This is almost a 
correct interpretation of the guesswork done in Algorithm [2J The following simple structural result 
will help articulate this notion in the proof of the main result. 

Lemma A. 2. IfC is solution to a Set Cover instance (X, S), then there is an ordering S±, . . . ,Sk 
of the cover-sets in C such that \Si \ U}=i Sj\ < jr f or an V 1 < i' < i < k. 

Proof. Order C greedily: Iteratively select the set in C that covers the most uncovered items. 
Then, by definition of this ordering, for the first i' sets we have n > | U}=i Sj\ — *' " \ U}=i 
Moreover, for i > i' , since SV is chosen instead of Si, we must have |Si\|J*- =i <%l — l<SV\U}=i "%l — 
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Algorithm 2 An Improved Approximation for Set Cover 



1: C<-5 

2: for each collection T> of at most d sets in S do 

3: Let T^{Sg5: \S\\J TeV T\ < §} 

4: if T U P cannot cover X then skip to the next iteration 

5: while T> does not cover X do 

6: Let 5 <r- arg min SeT \ s \{J^ v t\ 

7: V^VU{S} 

8: end while 

9: if V is cheaper than C then C T> 

10: end for 

11: return C 



We are now ready to give all details for the combinatorial proof of Theorem 11.11 

Proof of Theorem \l.l[ Let C* be an optimal solution of cost OPT and size, say, k. Furthermore, 
let SI, . . . , St be an ordering of the cover-sets in C* with the property guaranteed by Lemma IA.2I 
If k < d then the set C* will be considered in some iteration so Algorithm [2] will actually find an 
optimal solution. 

Otherwise, consider the iteration of the outer loop that guessed V = {S 1 *, . . . ,<S*j}. Consider 
the Set Cover instance (Y,T) with items Y := X \ \J TeV T and cover-sets T := {S \ \J TeT ,T : 
S G S, \ S \ \Jt£T>T\ < -|}- The cost of the cover-sets S G T in this instance should be equal to 
their original costs (before we subtracted Ure£>^)- Each cover-set in T has size at most ^ so, by 
Theorem lA.il the greedy algorithm applied to this instance is an i? n /rf- approximation. Lemma [A. 21 
guarantees that the restriction of the cover-sets S^ +1 , . . . , S£ to X \ \J Te -pT have size at most 
2 so these restrictions form a valid solution for (Y,T) of cost Y^l=d+i c (^i)- Thus, the optimum 
solution to the Set Cover instance (Y, T) has cost Yli=d+i c (^i)- Steps [MS] act just like the greedy 
algorithm on this instance (Y, T) which, by Theorem lA.il adds a total of H« ■ Yli=d+i c (^i) t° ^ ne 
cost of V. The overall cost of the final solution V is at most Yli=i C (S?) + H- ' J2i=d+i c (^t) — 

■ Ei=i c ( S i) = H z ■ 0PT - Finally, note that there are m ^ ' iterations of the outer loop, each 
taking polynomial time. □ 
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